

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">

<html class="pageview">


<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">

<base target="_top">
<style type="text/css">
  

/* default css */

table {
  font-size: 1em;
  line-height: inherit;
  border-collapse: collapse;
}


tr {
  
  text-align: left;
  
}


div, address, ol, ul, li, option, select {
  margin-top: 0px;
  margin-bottom: 0px;
}

p {
  margin: 0px;
}


pre {
  font-family: Courier New;
  white-space: pre-wrap;
  margin:0;
}

body {
  margin: 6px;
  padding: 0px;
  font-family: Verdana, sans-serif;
  font-size: 10pt;
  background-color: #ffffff;
  color: #000;
}


img {
  -moz-force-broken-image-icon: 1;
}

@media screen {
  html.pageview {
    background-color: #f3f3f3 !important;
    overflow-x: hidden;
    overflow-y: scroll;
  }

  

  body {
    min-height: 1100px;
    
    counter-reset: __goog_page__;
  }
  
  * html body {
    height: 1100px;
  }
  /* Prevent repaint errors when scrolling in Safari. This "Star-7" css hack
     targets Safari 3.1, but not WebKit nightlies and presumably Safari 4.
     That's OK because this bug is fixed in WebKit nightlies/Safari 4 :-). */
  html*#wys_frame::before {
    content: '\A0';
    position: fixed;
    overflow: hidden;
    width: 0;
    height: 0;
    top: 0;
    left: 0;
  }
  
  .pageview body {
    border-top: 1px solid #ccc;
    border-left: 1px solid #ccc;
    border-right: 2px solid #bbb;
    border-bottom: 2px solid #bbb;
    width: 648px !important;
    margin: 15px auto 25px;
    padding: 40px 50px;
  }
  /* IE6 */
  * html {
    overflow-y: scroll;
  }
  * html.pageview body {
    overflow-x: auto;
  }
  

  
    
    
    .writely-callout-data {
      display: inline-block;
      width: 1px;
      height: 1px;
      overflow: hidden;
      margin-left: -1px;
    }
    

    .writely-footnote-marker {
      background-image: url('MISSING');
      background-color: transparent;
      background-repeat: no-repeat;
      width: 7px;
      overflow: hidden;
      height: 16px;
      vertical-align: top;

      
      -moz-user-select: none;
    }
    .editor .writely-footnote-marker {
      cursor: move;
    }
    .writely-footnote-marker-highlight {
      background-position: -15px 0;
      -moz-user-select: text;
    }
    .writely-footnote-hide-selection ::-moz-selection, .writely-footnote-hide-selection::-moz-selection {
      background: transparent;
    }
    .writely-footnote-hide-selection ::selection, .writely-footnote-hide-selection::selection {
      background: transparent;
    }
    .writely-footnote-hide-selection {
      cursor: move;
    }

    /* Comments */
    .writely-comment-yellow {
      background-color: #ffffd7;
    }
    .writely-comment-orange {
      background-color: #ffe3c0;
    }
    .writely-comment-pink {
      background-color: #ffd7ff;
    }
    .writely-comment-green {
      background-color: #d7ffd7;
    }
    .writely-comment-blue {
      background-color: #d7ffff;
    }
    .writely-comment-purple {
      background-color: #eed7ff;
    }

  


  
  .br_fix span+br:not(:-moz-last-node) {
    
    position:relative;
    
    left: -1ex
    
  }

  
  #cb-p-tgt {
    font-size: 8pt;
    padding: .4em;
    background-color: #ddd;
    color: #333;
  }
  #cb-p-tgt-can {
    text-decoration: underline;
    color: #36c;
    font-weight: bold;
    margin-left: 2em;
  }
  #cb-p-tgt .spin {
    width: 16px;
    height: 16px;
    background: url(//ssl.gstatic.com/docs/clipboard/spin_16o.gif) no-repeat;
  }
}

h6 { font-size: 8pt }
h5 { font-size: 8pt }
h4 { font-size: 10pt }
h3 { font-size: 12pt }
h2 { font-size: 14pt }
h1 { font-size: 18pt }

blockquote {padding: 10px; border: 1px #DDD dashed }

.webkit-indent-blockquote { border: none; }

a img {border: 0}

.pb {
  border-width: 0;
  page-break-after: always;
  /* We don't want this to be resizeable, so enforce a width and height
     using !important */
  height: 1px !important;
  width: 100% !important;
}

.editor .pb {
  border-top: 1px dashed #C0C0C0;
  border-bottom: 1px dashed #C0C0C0;
}

div.google_header, div.google_footer {
  position: relative;
  margin-top: 1em;
  margin-bottom: 1em;
}


/* Table of contents */
.editor div.writely-toc {
  background-color: #f3f3f3;
  border: 1px solid #ccc;
}
.writely-toc > ol {
  padding-left: 3em;
  font-weight: bold;
}
ol.writely-toc-subheading {
  padding-left: 1em;
  font-weight: normal;
}
/* IE6 only */
* html writely-toc ol {
  list-style-position: inside;
}
.writely-toc-none {
  list-style-type: none;
}
.writely-toc-decimal {
  list-style-type: decimal;
}
.writely-toc-upper-alpha {
  list-style-type: upper-alpha;
}
.writely-toc-lower-alpha {
  list-style-type: lower-alpha;
}
.writely-toc-upper-roman {
  list-style-type: upper-roman;
}
.writely-toc-lower-roman {
  list-style-type: lower-roman;
}
.writely-toc-disc {
  list-style-type: disc;
}

/* Ordered lists converted to numbered lists can preserve ordered types, and
   vice versa. This is confusing, so disallow it */
ul[type="i"], ul[type="I"], ul[type="1"], ul[type="a"], ul[type="A"] {
  list-style-type: disc;
}

ol[type="disc"], ol[type="circle"], ol[type="square"] {
  list-style-type: decimal;
}

/* end default css */


  /* default print css */
  @media print {
    body {
      padding: 0;
      margin: 0;
    }

    div.google_header, div.google_footer {
      display: block;
      min-height: 0;
      border: none;
    }

    div.google_header {
      flow: static(header);
    }

    /* used to insert page numbers */
    div.google_header::before, div.google_footer::before {
      position: absolute;
      top: 0;
    }

    div.google_footer {
      flow: static(footer);
    }

    /* always consider this element at the start of the doc */
    div#google_footer {
      flow: static(footer, start);
    }

    span.google_pagenumber {
      content: counter(page);
    }

    span.google_pagecount {
      content: counter(pages);
    }

    .endnotes {
      page: endnote;
    }

    /* MLA specifies that endnotes title should be 1" margin from the top of the page. */
    @page endnote {
      margin-top: 1in;
    }

    callout.google_footnote {
      
      display: prince-footnote;
      footnote-style-position: inside;
      /* These styles keep the footnote from taking on the style of the text
         surrounding the footnote marker. They can be overridden in the
         document CSS. */
      color: #000;
      font-family: Times New Roman;
      font-size: 12.0pt;
      font-weight: normal;
    }

    /* Table of contents */
    #WritelyTableOfContents a::after {
      content: leader('.') target-counter(attr(href), page);
    }

    #WritelyTableOfContents a {
      text-decoration: none;
      color: black;
    }

    /* Comments */
    .writely-comment-yellow {
      background-color: #ffffd7;
    }
    .writely-comment-orange {
      background-color: #ffe3c0;
    }
    .writely-comment-pink {
      background-color: #ffd7ff;
    }
    .writely-comment-green {
      background-color: #d7ffd7;
    }
    .writely-comment-blue {
      background-color: #d7ffff;
    }
    .writely-comment-purple {
      background-color: #eed7ff;
    }
  }

  @page {
    @top {
      content: flow(header);
    }
    @bottom {
      content: flow(footer);
    }
    @footnotes {
      border-top: solid black thin;
      padding-top: 8pt;
    }
  }
  /* end default print css */


/* custom css */


/* end custom css */

/* ui edited css */

body {
  font-family: Times New Roman;
  
  font-size: 12.0pt;
  line-height: 1.5;
  background-color: #ffffff;
}
/* end ui edited css */


/* editor CSS */
.editor a:visited {color: #551A8B}
.editor table.zeroBorder {border: 1px dotted gray}
.editor table.zeroBorder td {border: 1px dotted gray}
.editor table.zeroBorder th {border: 1px dotted gray}


.editor div.google_header, .editor div.google_footer {
  border: 2px #DDDDDD dashed;
  position: static;
  width: 100%;
  min-height: 2em;
}

.editor .misspell {background-color: yellow}

.editor .writely-comment {
  font-size: 9pt;
  line-height: 1.4;
  padding: 1px;
  border: 1px dashed #C0C0C0
}


/* end editor CSS */

</style>

  
  <title>SemanticSQL Redux</title>

</head>

<body 
    
    >
    
    
    
<font size=3><b><u><font size=5>SemanticSQL Redux </font></u></b><br>
<font size=3>Matthew Fay and&nbsp;</font><font size=3>Jodyann Coley</font><br>
</font>
<div>
  <font size=3>Original Term Project, 6.863, Spring 2010</font><br>
</div>
<div>
  Matthew Fay
</div>
<div>
  Project Update, Summer 2010
</div>
<br>
<div>
  Current Code:&nbsp;<a href=http://semanticsql.dnsalias.org/SemanticSQL/SemanticSQL.zip>http://semanticsql.dnsalias.org/SemanticSQL/SemanticSQL.zip</a>
</div>
<div>
  Live Current Version:&nbsp;<a href=http://semanticsql.dnsalias.org/SemanticSQL/>http://semanticsql.dnsalias.org/SemanticSQL/</a>
</div>
<br>
<div>
  <font size=4><b><u>Introduction</u></b></font>
</div>
<font size=3>&nbsp;&nbsp;&nbsp;&nbsp; &nbsp;&nbsp; &nbsp;A large goal of modern computational studies is to develop intelligent agents that are able to reason. Intelligent systems must be equipped with tools that allow them to learn about their environments, process the information, and make decisions. &nbsp;From a human perspective, the root of such tools lies in language.&nbsp; Thus if we are to be able to enable computational reasoning, we must develop useful computational language models.&nbsp;</font>
<div>
  <font size=3>&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;Language research can be divided into a number of areas such as syntax, morphology and semantics.&nbsp; We decided to focus on semantics for our project.&nbsp; Semantics allows us to derive meaning from sentences in order to gather information from natural language descriptions. &nbsp;This project aims to create a semantic parser of English that is able to extract meaning from sentences and store the information about a scene into a SQL database. &nbsp;The same parser can then intelligently answer questions phrased in English about the constructed scene. &nbsp;Previous work in this area was done by Eric Lieberman and Mike Lin. &nbsp;The goal of our work is to improve upon the parsing methods and then to take it to the next level by incorporating new levels of understanding.&nbsp; Specifically, we sought to incorporate the following grammar constructs:</font>
</div>
<ol>
  <ol>
    <li>
      <font size=3>How an action was taken using adverbs</font>
    </li>
    <li>
      <font size=3>Location of events using prepositional phrases</font>
    </li>
    <li>
      <font size=3>Instrument usage using prepositional phrases</font>
    </li>
    <li>
      <font size=3>Cause and Effect using If-Then sentence structures</font>
    </li>
    <li>
      <font size=3>Usage of the adverb not for negations</font>
    </li>
    <li>
      Passive sentences
    </li>
    <li>
      Object classes and quantifiers
    </li>
  </ol>
</ol>
<font size=3>Additionally, new question structures needed to be incorporated to allow this new information to be used.&nbsp; Finally, we wanted to improve the semantic processing to allow more intelligent question responses such as through improved indefinite noun usage.&nbsp; For example, if the sentence "John likes the cats" was previously submitted to the system, then the system should return yes to the question "Does John like a cat".<br>
&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;With the expansions incorporated, the SemanticSQL system can understand sentences such as "John happily ate icecream with a spork in the park" and then appropriately answer questions such as "How did John eat". &nbsp;The current version currently is running on a test server and can be found <a href=http://semanticsql.dnsalias.org/SemanticSQL id=xf:x title=http://semanticsql.dnsalias.org/SemanticSQL>here</a>.<br>
<br>
<br>
<font size=4><b><u>Design and Implementation</u></b></font></font><font size=3><br>
The SemanticSQL processor works through the following steps:</font>
<ol>
  <ol>
    <li>
      <font size=3>The English sentence is passed through a grammar parser.</font>
    </li>
    <li>
      <font size=3>The parser pulls the semantic meaning from the sentence using lambda calculus.</font>
    </li>
    <li>
      <font size=3>The semantic event is translated into the appropriate SQL statements.</font>
    </li>
  </ol>
</ol>
<font size=3>&nbsp;A simple example of this process is the question "Did John sleep", which would be translated into a SQL statement resembling "SELECT FROM Event WHERE action=sleep and agent=john". &nbsp;The current implementation, however, allows more robust object tracking. &nbsp;Whenever a new agent takes an action in the scene, the system checks to see if it knows of the object using a SELECT statement, then adds the object to the database using an INSERT statement if it is not found.&nbsp; This allows an object to appropriately be involved in multiple events. &nbsp;While this makes the system more flexible, it also means that even simple statements such as "John kissed Mary" requires 3-5 SQL statements depending on whether or not John and Mary are already in the scene.<br>
</font>
<div>
  <font size=3>&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;The database currently consists of three tables for semantic knowledge: &nbsp;Events, Objects, and Modifiers. &nbsp;The table structures are detailed below.</font>
</div>
<font size=3><br>
</font>
<table border=0 bordercolor=#000000 cellpadding=3 cellspacing=0 class=zeroBorder id=k3r6 width=100%>
  <tbody>
  <tr>
    <td width=100%>
      Modifier Table<br>
      <table border=1 bordercolor=#000000 cellpadding=3 cellspacing=0 id=tnx3>
        <tbody>
        <tr>
          <td>
            <font size=3>Mod</font>
          </td>
          <td>
            <font size=3>The adjective, ex. red, green</font>
          </td>
        </tr>
        <tr>
          <td>
            <font size=3>Target</font>
          </td>
          <td>
            <font size=3>The ID of the object modified</font>
          </td>
        </tr>
        </tbody>
      </table>
    </td>
  </tr>
  <tr>
    <td width=100%>
      <font size=3>Object Table<br>
      </font>
      <table border=1 bordercolor=#000000 cellpadding=3 cellspacing=0 id=qirw>
        <tbody>
        <tr>
          <td>
            <font size=3>ID</font>
          </td>
          <td>
            <font size=3>Unique ID Assigned to Object</font>
          </td>
        </tr>
        <tr>
          <td>
            <font size=3>Name</font>
          </td>
          <td>
            <font size=3>The name of the object, ex: John, Mary</font>
          </td>
        </tr>
        <tr>
          <td>
            <font size=3>Type</font>
          </td>
          <td>
            <font size=3>The type of the object, ex: cat, dogs</font>
          </td>
        </tr>
        <tr>
          <td>
            <font size=3>Definite</font>
          </td>
          <td>
            <font size=3>Whether the object is definite, indefinite, or modified by a quantifier</font>
          </td>
        </tr>
        </tbody>
      </table>
      <font size=3><br>
      </font>
    </td>
  </tr>
  <tr>
    <td width=100%>
      <font size=3>Event Table</font>
      <table border=1 bordercolor=#000000 cellpadding=3 cellspacing=0 id=r-ia>
        <tbody>
        <tr>
          <td>
            <font size=3>Action</font>
          </td>
          <td>
            <font size=3>The action taken by the agent, ex: eat, sleep</font>
          </td>
        </tr>
        <tr>
          <td>
            <font size=3>Agent</font>
          </td>
          <td>
            <font size=3>The ID of the object that does the action</font>
          </td>
        </tr>
        <tr>
          <td>
            <font size=3>Patient</font>
          </td>
          <td>
            <font size=3>The ID of the direct object</font>
          </td>
        </tr>
        <tr>
          <td>
            <font size=3>Beneficiary</font>
          </td>
          <td>
            <font size=3>The ID of the recipient of the action, for example who something was given to</font>
          </td>
        </tr>
        <tr>
          <td>
            <font size=3>Tense</font>
          </td>
          <td>
            <font size=3>When the event occurred</font>
          </td>
        </tr>
        <tr>
          <td>
            <font size=3>Loc_Prep</font>
          </td>
          <td>
            <font size=3>The preposition related to the location, ex: in, under<br>
            </font>
          </td>
        </tr>
        <tr>
          <td>
            <font size=3>Loc_Target</font>
          </td>
          <td>
            <font size=3>The ID of the location where the event took place<br>
            </font>
          </td>
        </tr>
        <tr>
          <td>
            <font size=3>Instrument</font>
          </td>
          <td>
            <font size=3>The ID of the instrument used in the event</font>
          </td>
        </tr>
        <tr>
          <td>
            <font size=3>Adverb</font>
          </td>
          <td>
            <font size=3>How the action was taken</font>
          </td>
        </tr>
        <tr>
          <td>
            Adjective
          </td>
          <td>
            Adjectives used with the verb 'to be'
          </td>
        </tr>
        <tr>
          <td>
            Negation
          </td>
          <td>
            Whether an event is a truth or a negation
          </td>
        </tr>
        </tbody>
      </table>
      <font size=3><br>
      </font>
    </td>
  </tr>
  </tbody>
</table>
<font size=3><br>
The following is a sample set of sentences that have been annotated with the generated SQL code to show how the system actually works.</font><br>
<br>
<div>
  <font size=3><u>Sentence:</u> <i>Poirot likes the dogs</i><br>
  <br>
  </font>
</div>
<table border=1 cellpadding=1 cellspacing=2 class=log>
  <tbody>
  <tr>
    <th>
      <font size=3>Step</font>
    </th>
    <th>
      <font size=3>Description</font>
    </th>
    <th>
      <font size=3>SQL</font>
    </th>
    <th>
      <font size=3>Result</font>
    </th>
  </tr>
  <tr>
    <td>
      <center><font size=3>1</font></center>
    </td>
    <td>
      <font size=3>resolving named object</font>
    </td>
    <td>
      <font size=3>SELECT id FROM Object WHERE name='Poirot'</font>
    </td>
    <td>
      <font size=3>not found, will insert</font>
    </td>
  </tr>
  <tr>
    <td>
      <center><font size=3>2</font></center>
    </td>
    <td>
      <font size=3>inserting a new named object</font>
    </td>
    <td>
      <font size=3>INSERT INTO Object(`name`,`type`,`definite`) VALUES ('Poirot',NULL,NULL)</font>
    </td>
    <td>
      <font size=3>OK, id = 184</font>
    </td>
  </tr>
  <tr>
    <td>
      <center><font size=3>3</font></center>
    </td>
    <td>
      <font size=3>resolving unnamed object</font>
    </td>
    <td>
      <font size=3>SELECT id FROM Object WHERE `type`='dogs' AND Object.definite = '+'</font>
    </td>
    <td>
      <font size=3>not found, will insert</font>
    </td>
  </tr>
  <tr>
    <td>
      <center><font size=3>4</font></center>
    </td>
    <td>
      <font size=3>inserting a new unnamed object</font>
    </td>
    <td>
      <font size=3>INSERT INTO Object(`name`,`type`,`definite`) VALUES (NULL,'dogs','+')</font>
    </td>
    <td>
      <font size=3>OK, id = 185</font>
    </td>
  </tr>
  <tr>
    <td>
      <center><font size=3>5</font></center>
    </td>
    <td>
      <font size=3>adding fact</font>
    </td>
    <td>
      <font size=3>INSERT INTO Event VALUES ('like',184,185,NULL,'present',NULL,NULL,NULL,NULL)</font>
    </td>
    <td>
      <font size=3>OK</font>
    </td>
  </tr>
  </tbody>
</table>
<br>
<div>
  <font size=3>SemanticSQL now knows that "Poirot likes the dogs".&nbsp; Similarly, we can say that "Mary likes the cats".&nbsp; After these sentences are submitted to the system, the system generates the following SQL code for the sentences below.<br>
  </font>
</div>
<br>
<div>
  <font size=3><u>Sentence:</u> <i>Does Poirot like the dogs<br>
  <br>
  </i></font>
  <table border=1 cellpadding=1 cellspacing=2 class=log>
    <tbody>
    <tr>
      <th>
        <font size=3><font face="'Times New Roman'">Step</font></font>
      </th>
      <th>
        <font size=3><font face="'Times New Roman'">Description</font></font>
      </th>
      <th>
        <font size=3><font face="'Times New Roman'">SQL</font></font>
      </th>
      <th>
        <font size=3><font face="'Times New Roman'">Result</font></font>
      </th>
    </tr>
    <tr>
      <td>
        <center><font size=3><font face="'Times New Roman'">1</font></font></center>
      </td>
      <td>
        <font size=3><font face="'Times New Roman'">resolving named object</font></font>
      </td>
      <td>
        <font size=3><font face="'Times New Roman'">SELECT id FROM Object WHERE name='poirot'</font></font>
      </td>
      <td>
        <font size=3><font face="'Times New Roman'">OK, id = 189</font></font>
      </td>
    </tr>
    <tr>
      <td>
        <center><font size=3><font face="'Times New Roman'">2</font></font></center>
      </td>
      <td>
        <font size=3><font face="'Times New Roman'">resolving unnamed object</font></font>
      </td>
      <td>
        <font size=3><font face="'Times New Roman'">SELECT id FROM Object WHERE `type`='dogs' AND Object.definite = '+'</font></font>
      </td>
      <td>
        <font size=3><font face="'Times New Roman'">OK, id = 190</font></font>
      </td>
    </tr>
    <tr>
      <td>
        <center><font size=3><font face="'Times New Roman'">3</font></font></center>
      </td>
      <td>
        <font size=3><font face="'Times New Roman'">yes/no question</font></font>
      </td>
      <td>
        <font size=3><font face="'Times New Roman'">SELECT CASE WHEN COUNT(*) &gt; 0 THEN 'Yes' ELSE 'No' END FROM Event WHERE action='like' AND agent=189 AND patient=190 AND tense='present'</font></font>
      </td>
      <td>
        <font size=3><font face="'Times New Roman'">Yes</font></font>
      </td>
    </tr>
    </tbody>
  </table>
  <br>
  <br>
  <div>
    <font size=3><u>Sentence:</u> <i>Does Poirot like the cats<br>
    <br>
    </i></font>
  </div>
  <table border=1 cellpadding=1 cellspacing=2 class=log>
    <tbody>
    <tr>
      <th>
        <font size=3>Step</font>
      </th>
      <th>
        <font size=3>Description</font>
      </th>
      <th>
        <font size=3>SQL</font>
      </th>
      <th>
        <font size=3>Result</font>
      </th>
    </tr>
    <tr>
      <td>
        <center><font size=3>1</font></center>
      </td>
      <td>
        <font size=3>resolving named object</font>
      </td>
      <td>
        <font size=3>SELECT id FROM Object WHERE name='poirot'</font>
      </td>
      <td>
        <font size=3>OK, id = 189</font>
      </td>
    </tr>
    <tr>
      <td>
        <center><font size=3>2</font></center>
      </td>
      <td>
        <font size=3>resolving unnamed object</font>
      </td>
      <td>
        <font size=3>SELECT id FROM Object WHERE `type`='cats' AND Object.definite = '+'</font>
      </td>
      <td>
        <font size=3>OK, id = 194</font>
      </td>
    </tr>
    <tr>
      <td>
        <center><font size=3>3</font></center>
      </td>
      <td>
        <font size=3>yes/no question</font>
      </td>
      <td>
        <font size=3>SELECT CASE WHEN COUNT(*) &gt; 0 THEN 'Yes' ELSE 'No' END FROM Event WHERE action='like' AND agent=189 AND patient=194 AND tense='present'</font>
      </td>
      <td>
        <font size=3>No</font>
      </td>
    </tr>
    </tbody>
  </table>
  <br>
</div>
<div>
  <font size=3>S</font><font size=3>emanticSQL correctly detects that Poirot likes the dogs and is not confused by the addition of the cats to the scene.&nbsp; The following sections discuss the implementation of the new features and grammar constructs available in SemanticSQL.</font><br>
  <br>
  <div>
    <b><u><font size=3>Adverbs</font></u></b><font size=3><br>
    &nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;The system allows users to add adverbs to modify verbs, as demonstrated in the sample sequence of statements in following table.&nbsp;<br>
    </font><b><u><font size=3><br>
    </font></u></b>
  </div>
  <table border=1 bordercolor=#000000 cellpadding=3 cellspacing=0 id=hfp4 width=100%>
    <tbody>
    <tr>
      <td width=50%>
        <font size=3><b>Sentence<br>
        </b></font>
      </td>
      <td width=50%>
        <font size=3><b>System Response<br>
        </b></font>
      </td>
    </tr>
    <tr>
      <td width=50%>
        John eats quickly<br>
      </td>
      <td width=50%>
        OK<br>
      </td>
    </tr>
    <tr>
      <td width=50%>
        How does John eat<br>
      </td>
      <td width=50%>
        quickly<br>
      </td>
    </tr>
    <tr>
      <td width=50%>
        John eats gracefully<br>
      </td>
      <td width=50%>
        OK<br>
      </td>
    </tr>
    <tr>
      <td valign=top>
        How does John eat<br>
      </td>
      <td valign=top>
        quickly and gracefully<br>
      </td>
    </tr>
    </tbody>
  </table>
  <font size=3><br>
  </font>
  <div>
    <font size=3><b><u>Locations</u></b></font>
  </div>
  <font size=3>&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;Our system allows users to specify the locations of events and ask questions regarding where events occur.&nbsp; In our design, locations are considered to be features of verbs.&nbsp; The location feature consists of the preposition of the prepositional phrase (Loc_Prep in the Event Table) and the object of the location (Loc_Target in the Event Table).&nbsp; The table below illustrates an example sequence of sentences that are allowed in the system and the system's responses.<br>
  <br>
  </font>
  <table border=1 bordercolor=#000000 cellpadding=3 cellspacing=0 id=jkrp width=100%>
    <tbody>
    <tr>
      <td width=50%>
        <font size=3><b>Sentence<br>
        </b></font>
      </td>
      <td width=50%>
        <font size=3><b>System Response<br>
        </b></font>
      </td>
    </tr>
    <tr>
      <td width=50%>
        <font size=3>John kissed Mary in the park<br>
        </font>
      </td>
      <td width=50%>
        <font size=3>OK<br>
        </font>
      </td>
    </tr>
    <tr>
      <td width=50%>
        <font size=3>Where did John kiss Mary<br>
        </font>
      </td>
      <td width=50%>
        <font size=3>in the park<br>
        </font>
      </td>
    </tr>
    <tr>
      <td width=50%>
        <font size=3>John happily kissed Mary under the table<br>
        </font>
      </td>
      <td width=50%>
        <font size=3>OK<br>
        </font>
      </td>
    </tr>
    <tr>
      <td width=50%>
        <font size=3>Where did John kiss Mary<br>
        </font>
      </td>
      <td width=50%>
        <font size=3>in the park and under the table<br>
        </font>
      </td>
    </tr>
    <tr>
      <td width=50%>
        <font size=3>Where did John happily kiss Mary<br>
        </font>
      </td>
      <td width=50%>
        <font size=3>under the table<br>
        </font>
      </td>
    </tr>
    </tbody>
  </table>
  <font size=3><br>
  </font>
  <div>
    <font size=3><b><u>Instruments</u></b></font>
  </div>
  <font size=3>&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;The system allows users to specify instruments that are used during events.&nbsp; Similar to locations, instruments are consi<font size=3>d</font>ere<font size=3>d</font> as features of verbs.&nbsp; An example sequence is shown in the table below.<br>
  <br>
  </font>
  <table border=1 bordercolor=#000000 cellpadding=3 cellspacing=0 id=sw5y width=100%>
    <tbody>
    <tr>
      <td width=50%>
        <font size=3><b>Sentence<br>
        </b></font>
      </td>
      <td width=50%>
        <font size=3><b>System Response<br>
        </b></font>
      </td>
    </tr>
    <tr>
      <td width=50%>
        John ate icecream with a spoon<br>
      </td>
      <td width=50%>
        OK<br>
      </td>
    </tr>
    <tr>
      <td width=50%>
        What <font size=3>d</font>i<font size=3>d</font> John eat with<br>
      </td>
      <td width=50%>
        a spoon<br>
      </td>
    </tr>
    <tr>
      <td width=50%>
        John ate the chicken with a fork<br>
      </td>
      <td width=50%>
        OK<br>
      </td>
    </tr>
    <tr>
      <td valign=top>
        What <font size=3>d</font>i<font size=3>d</font> John eat the chicken with<br>
      </td>
      <td valign=top>
        a fork<br>
      </td>
    </tr>
    <tr>
      <td width=50%>
        What <font size=3>d</font>i<font size=3>d</font> John eat with<br>
      </td>
      <td width=50%>
        a spoon an<font size=3>d</font> a fork<br>
      </td>
    </tr>
    <tr>
      <td valign=top>
        John slept happily in a be<font size=3>d</font> with a pillow<br>
      </td>
      <td valign=top>
        OK<br>
      </td>
    </tr>
    <tr>
      <td valign=top>
        What <font size=3>d</font>i<font size=3>d</font> John sleep with<br>
      </td>
      <td valign=top>
        a pillow<br>
      </td>
    </tr>
    <tr>
      <td valign=top>
        John slept in the be<font size=3>d</font> with Mary<br>
      </td>
      <td valign=top>
        OK<br>
      </td>
    </tr>
    <tr>
      <td valign=top>
        What <font size=3>d</font>i<font size=3>d</font> John sleep with<br>
      </td>
      <td valign=top>
        a pillow an<font size=3>d</font> Mary<br>
      </td>
    </tr>
    </tbody>
  </table>
  <font size=3><br>
  </font>
  <div>
    <font size=3><b><u>Cause and Effect</u></b></font>
  </div>
  <font size=3>&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;Users are able to a<font size=3>d</font><font size=3>d</font> new events to the <font size=3>d</font>atabase through if-then statements.&nbsp; If the first part of the statement (the cause) is true base<font size=3>d</font> on the information in the <font size=3>d</font>atabase, then the secon<font size=3>d</font> part of the statement (the effect) is a<font size=3>d</font><font size=3>d</font>e<font size=3>d</font> to the&nbsp;<font size=3>d</font>atabase.&nbsp; Otherwise, "the effect" is not a<font size=3>d</font><font size=3>d</font>e<font size=3>d</font> to the&nbsp;<font size=3>d</font>atabase.&nbsp; Some example sentences are in the table below. &nbsp;<br>
  <br>
  </font>
  <table border=1 bordercolor=#000000 cellpadding=3 cellspacing=0 id=bse7 width=100%>
    <tbody>
    <tr>
      <td width=50%>
        <font size=3><b>Sentence<br>
        </b></font>
      </td>
      <td width=50%>
        <font size=3><b>System Response<br>
        </b></font>
      </td>
    </tr>
    <tr>
      <td width=50%>
        Mary likes John<br>
      </td>
      <td width=50%>
        OK<br>
      </td>
    </tr>
    <tr>
      <td width=50%>
        If Mary likes John then Mary likes the werewolves<br>
      </td>
      <td width=50%>
        Cause was foun<font size=3>d</font>, Effect a<font size=3>d</font><font size=3>d</font>e<font size=3>d</font> to the <font size=3>d</font>atabase<br>
      </td>
    </tr>
    <tr>
      <td width=50%>
        If Mary likes the cats then Mary likes the <font size=3>d</font>ogs<br>
      </td>
      <td width=50%>
        Cause not foun<font size=3>d</font>, Effect will not be a<font size=3>d</font><font size=3>d</font>e<font size=3>d</font><br>
      </td>
    </tr>
    <tr>
      <td valign=top>
        John sleeps happily<br>
      </td>
      <td valign=top>
        OK<br>
      </td>
    </tr>
    <tr>
      <td width=50%>
        If John sleeps happily then John likes Mary<br>
      </td>
      <td width=50%>
        Cause was foun<font size=3>d</font>, Effect a<font size=3>d</font><font size=3>d</font>e<font size=3>d</font> to the <font size=3>d</font>atabase<br>
      </td>
    </tr>
    </tbody>
  </table>
  <font size=3><br>
  </font>
</div>
<div>
  <font size=3><b><u>Negations</u></b></font>
</div>
<div>
  <font size=3>&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;Negations can be applied to nearly any sentence structure that exists in the grammar. &nbsp;Some examples include: &nbsp;John does not like Mary, &nbsp;Who did not eat the icecream, and &nbsp;Poirot did not give the dog to John. &nbsp;The adverb not is applied on the event level tracking whether an event happened or whether the event specifically did not occur.</font>
</div>
<font size=3><br>
</font>
<table border=1 bordercolor=#000000 cellpadding=3 cellspacing=0 id=gkcl width=100%>
  <tbody>
  <tr style=TEXT-ALIGN:left>
    <td width=50%>
      <font size=3><b>Sentence<br>
      </b></font>
    </td>
    <td width=50%>
      <font size=3><b>System Response<br>
      </b></font>
    </td>
  </tr>
  <tr style=TEXT-ALIGN:left>
    <td width=50%>
      <font size=3>Mary likes John<br>
      </font>
    </td>
    <td width=50%>
      <font size=3>OK<br>
      </font>
    </td>
  </tr>
  <tr style=TEXT-ALIGN:left>
    <td width=50%>
      <font size=3>Mary does not like Poirot<br>
      </font>
    </td>
    <td width=50%>
      <font size=3>OK</font>
    </td>
  </tr>
  <tr style=TEXT-ALIGN:left>
    <td width=50%>
      <font size=3>Does Mary like John</font>
    </td>
    <td width=50%>
      <font size=3>Yes</font>
    </td>
  </tr>
  <tr style=TEXT-ALIGN:left>
    <td valign=top>
      <font size=3>Who does Mary not like<br>
      </font>
    </td>
    <td valign=top>
      <font size=3>Poirot</font>
    </td>
  </tr>
  <tr style=TEXT-ALIGN:left>
    <td width=50%>
      <font size=3>Does Mary like Poirot<br>
      </font>
    </td>
    <td width=50%>
      <font size=3>No</font>
    </td>
  </tr>
  </tbody>
</table>
<font size=3><br>
</font><b><u>Passive Sentences and the Verb "to be"<br>
</u></b>
<div>
  <font size=3>&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;The verb "to be" adds an additional layer of complexity into the English grammar. &nbsp;Primarily it allows numerous new sentence constructions to be included. &nbsp;Passive constructions such as John was driven by Poirot and Mary was liked by the dogs can now be interpreted correctly and stored in the scene database. &nbsp;Additionally constructs such as John was happy and Mary is blue were enabled by including an adjective portion of an event occurrence. &nbsp;A final category included is the addition of the ability to ascribe classes to proper nouns which will be described more in the next section.</font>
</div>
<font size=3><br>
</font>
<table border=1 bordercolor=#000000 cellpadding=3 cellspacing=0 id=xlzw width=100%>
  <tbody>
  <tr style=TEXT-ALIGN:left>
    <td width=50%>
      <font size=3><b>Sentence<br>
      </b></font>
    </td>
    <td width=50%>
      <font size=3><b>System Response<br>
      </b></font>
    </td>
  </tr>
  <tr style=TEXT-ALIGN:left>
    <td width=50%>
      <font size=3>Mary was liked by John<br>
      </font>
    </td>
    <td width=50%>
      <font size=3>OK<br>
      </font>
    </td>
  </tr>
  <tr style=TEXT-ALIGN:left>
    <td width=50%>
      <font size=3>Does John like Mary<br>
      </font>
    </td>
    <td width=50%>
      <font size=3>Yes</font>
    </td>
  </tr>
  <tr style=TEXT-ALIGN:left>
    <td width=50%>
      <font size=3>Poirot ate icecream</font>
    </td>
    <td width=50%>
      <font size=3>OK</font>
    </td>
  </tr>
  <tr style=TEXT-ALIGN:left>
    <td valign=top>
      <font size=3>Was the icecream eaten<br>
      </font>
    </td>
    <td valign=top>
      <font size=3>Yes</font>
    </td>
  </tr>
  <tr style=TEXT-ALIGN:left>
    <td width=50%>
      <font size=3>Who was the icecream eaten by<br>
      </font>
    </td>
    <td width=50%>
      <font size=3>Poirot</font>
    </td>
  </tr>
  </tbody>
</table>
<font size=3><br>
</font>
<div>
  <font size=3><b><u>Object Classes and Quantifiers</u></b><br>
  </font><font size=3>&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;The ability for people to categorize people, places and things is extremely important for our overall understanding of the world. &nbsp;Incorporated into the SemanticSQL grammar and SQL code is the new understanding of categories. &nbsp;The most simple of example of this is the following.</font>
</div>
<font size=3><br>
</font>
<table border=1 bordercolor=#000000 cellpadding=3 cellspacing=0 id=bpst width=100%>
  <tbody>
  <tr style=TEXT-ALIGN:left>
    <td width=50%>
      <font size=3><b>Sentence<br>
      </b></font>
    </td>
    <td width=50%>
      <font size=3><b>System Response<br>
      </b></font>
    </td>
  </tr>
  <tr style=TEXT-ALIGN:left>
    <td width=50%>
      <font size=3>John is a person</font>
    </td>
    <td width=50%>
      <font size=3>OK<br>
      </font>
    </td>
  </tr>
  <tr style=TEXT-ALIGN:left>
    <td width=50%>
      Is John happy
    </td>
    <td width=50%>
      No
    </td>
  </tr>
  <tr style=TEXT-ALIGN:left>
    <td width=50%>
      <font size=3>Everyone is happy</font>
    </td>
    <td width=50%>
      <font size=3>OK</font>
    </td>
  </tr>
  <tr style=TEXT-ALIGN:left>
    <td width=50%>
      <font size=3>Is John happy</font>
    </td>
    <td width=50%>
      <font size=3>Yes</font>
    </td>
  </tr>
  </tbody>
</table>
<font size=3><br>
</font>
<div>
  <font size=3>&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;This ability can allow for a great deal of common knowledge understanding to begin to be incorporated into the SemanticSQL architecture. &nbsp;Quantifiers can also be successfully parsed and stored in the database.<br>
  </font><font size=3><br>
  </font>
</div>
<table border=1 bordercolor=#000000 cellpadding=3 cellspacing=0 id=v30c width=100%>
  <tbody>
  <tr style=TEXT-ALIGN:left>
    <td width=50%>
      <font size=3><b>Sentence<br>
      </b></font>
    </td>
    <td width=50%>
      <font size=3><b>System Response<br>
      </b></font>
    </td>
  </tr>
  <tr style=TEXT-ALIGN:left>
    <td width=50%>
      <font size=3>All people like dogs</font>
    </td>
    <td width=50%>
      <font size=3>OK<br>
      </font>
    </td>
  </tr>
  <tr style=TEXT-ALIGN:left>
    <td width=50%>
      <font size=3>Some people are happy</font>
    </td>
    <td width=50%>
      <font size=3>OK</font>
    </td>
  </tr>
  <tr style=TEXT-ALIGN:left>
    <td width=50%>
      <font size=3>Dogs eat some icecream</font>
    </td>
    <td width=50%>
      <font size=3>OK</font>
    </td>
  </tr>
  </tbody>
</table>
<font size=3><br>
</font>
<div>
  <font size=3>&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;Some quantifiers can also be extended into definite predictions about the scene that are not explicitly stated. &nbsp;Here's another more complex example of the power of quantifiers in classification.</font>
</div>
<font size=3><br>
</font>
<table border=1 bordercolor=#000000 cellpadding=3 cellspacing=0 id=loch width=100%>
  <tbody>
  <tr style=TEXT-ALIGN:left>
    <td width=50%>
      <font size=3><b>Sentence<br>
      </b></font>
    </td>
    <td width=50%>
      <font size=3><b>System Response<br>
      </b></font>
    </td>
  </tr>
  <tr style=TEXT-ALIGN:left>
    <td width=50%>
      <font size=3>John is a person</font>
    </td>
    <td width=50%>
      <font size=3>OK<br>
      </font>
    </td>
  </tr>
  <tr style=TEXT-ALIGN:left>
    <td width=50%>
      <font size=3>All people like dogs</font>
    </td>
    <td width=50%>
      <font size=3>OK</font>
    </td>
  </tr>
  <tr style=TEXT-ALIGN:left>
    <td width=50%>
      <font size=3>Does John like dogs</font>
    </td>
    <td width=50%>
      <font size=3>Yes</font>
    </td>
  </tr>
  <tr style=TEXT-ALIGN:left>
    <td width=50%>
      <font size=3>Fido is happy</font>
    </td>
    <td width=50%>
      <font size=3>OK</font>
    </td>
  </tr>
  <tr style=TEXT-ALIGN:left>
    <td width=50%>
      <font size=3>Does John like Fido</font>
    </td>
    <td width=50%>
      <font size=3>No</font>
    </td>
  </tr>
  <tr style=TEXT-ALIGN:left>
    <td width=50%>
      <font size=3>Fido is a dog</font>
    </td>
    <td width=50%>
      <font size=3>OK</font>
    </td>
  </tr>
  <tr style=TEXT-ALIGN:left>
    <td width=50%>
      <font size=3>Does John like Fido</font>
    </td>
    <td width=50%>
      <font size=3>Yes</font>
    </td>
  </tr>
  </tbody>
</table>
<font size=3><br>
</font><font size=3><br>
</font>
<div>
  <div>
    <font size=3><b><u>Infrastructure</u></b></font>
  </div>
  <font size=3>&nbsp;&nbsp;&nbsp; &nbsp;&nbsp; &nbsp;In addition to the features added to the grammar that have been covered thus far, a number of improvements were made to the infrastructure of SemanticSQL to improve its robustness.&nbsp; The first change was the construction of vocabulary tables in the database. &nbsp;The noun, verb, adjective, and adverb lexicons were moved out of the python server code into the SQL database. &nbsp;By making this change, new vocabulary can be instantly added to the dictionary without needing to change any </font><font size=3>of the underlying code. &nbsp;This makes dealing with new words very convenient. &nbsp;The adverb and adjective tables are simple word lists. &nbsp;The verb table stores the verb and its morphological forms.&nbsp;<br>
  &nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;The noun table additionally incorporates useful metadata.&nbsp; It stores not only each noun in the lexicon, but also the associated feature information such as whether the word is plural, a mass noun, or a name. &nbsp;Since the features are stored as text, they can easily be expanded on to fit the needs of a more powerful grammar without needing the noun table to be altered. &nbsp;Finally the noun table also stores the plural form of singular nouns, creating a semantic link between both forms of the noun.</font>
</div>
<div>
  <font size=3>&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;The noun lexicon table enabled us to deal with indefinite noun phrases in a more powerful way. &nbsp;For example, if you tell the system "Mary likes the dogs",&nbsp; then it will create an entry in the object table for Mary and another for a definite type "dogs". &nbsp;Later one might ask, "Does Mary like a dog". &nbsp;Intuitively the answer must be yes. &nbsp;While a simple query would return no, we can use the metadata linking the plural and singular noun forms to perform a second search. &nbsp;Now it will find that Mary likes more than one dog so she must like at least one dog, and the query returns yes.</font>
</div>
<div>
  <font size=3>&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;In order to facilitate some of these changes such as the incorporation of object classes, the SQL code generation it self had to be tweaked significantly. &nbsp;The primary change is that when asking questions that take classifiers into account, the SQL code is generated to allow more flexibility in the WHERE clauses including searching for multiple possible ID matches in the same search.<br>
  <br>
  </font>
</div>
<font size=3><br>
</font>
<div>
  <font size=3><font size=4><b><u>Testing and Analysis</u></b></font><br>
  </font>
</div>
<div>
  <font size=3>&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;During the course of development, we periodically tested the system using a large set of sentences that worked on the previous version of SemanticSQL, as well as new sample sentences that incorporated the features under active development. &nbsp;By doing this, we quickly identified problems that arose from the incorporation of new grammar, semantic, and SQL features and made sure that all previous constructs remained viable.&nbsp; </font><font size=3>Through our careful analysis of the grammar and overall SemanticSQL system, we discovered some problem areas in doing English to SQL translations.</font>
</div>
<div>
  <font size=3>&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;One area that presents a significant amount of difficulty are indefinite and anonymous objects. &nbsp;Part of this issue was addressed by incorporating more robust infrastructure, as previously mentioned. &nbsp;However, it still poses a significant problem. &nbsp;For example, the sentence "Mary likes the dogs" works fine, while "Mary likes dogs" does not. &nbsp;The problem goes beyond the syntax. &nbsp;Since the SemanticSQL database was designed to store information about a scene, it does not currently have the constructs to support detailed object descriptions such as Mary's general likes and dislikes that do not refer to specific objects in the scene, as demonstrated in the previous example. &nbsp;A temporary fix could have been implemented to tag "dogs" as indefinite in the failed sentence and add it to the object table. &nbsp;However, it's quite reasonable to assume that while "Mary likes dogs (in general)", that does not necessarily imply that "Mary likes the dogs (in the scene)". &nbsp;A possible solution could be to incorporate more detailed object descriptions and the potentially probabilistic scene understanding. &nbsp;In this case, if "Mary likes dogs", then there would be a high probability that "Mary likes the dogs".</font>
</div>
<div>
  <font size=3>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &nbsp;Another issue that arose as we incorporated an ever growing feature set was the problem of permutations.&nbsp; Take the sentence "John peacefully ate the fish in the park with a fork" for example.&nbsp; The same sentence could be expressed in a number of other ways such as "John ate the fish peacefully with a fork in the park", "John ate the fish peacefully in the park with a fork", "John ate the fish with a fork peacefully in the park", etc.&nbsp; We have incorporated many of these permutations.&nbsp; However, as we increase the number of features, it becomes more difficult to use the grammar.&nbsp; A related problem would be detecting whether or not the prepositions are attached to the verb as we assume, or if it would be more accurate to attach them to an object that appears earlier in the sentence.</font>
</div>
<div>
  <font size=3>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &nbsp;For convenience, the table below summarizes the current ability of the system in dealing with these issues.</font>
</div>
<div>
  <font size=3>&nbsp;<br>
  </font>
  <table border=1 bordercolor=#000000 cellpadding=3 cellspacing=0 id=upc4 width=100%>
    <tbody>
    <tr>
      <td valign=top>
        <b>Sentence<br>
        </b>
      </td>
      <td valign=top>
        <b>System Response<br>
        </b>
      </td>
    </tr>
    <tr>
      <td width=50%>
        Mary likes the dogs
      </td>
      <td width=50%>
        OK
      </td>
    </tr>
    <tr>
      <td width=50%>
        Mary likes dogs
      </td>
      <td width=50%>
        NOT OK
      </td>
    </tr>
    <tr>
      <td width=50%>
        John peacefully ate the fish in the park with a fork
      </td>
      <td width=50%>
        OK
      </td>
    </tr>
    <tr>
      <td valign=top>
        John ate the fish peacefully with a fork in the park<br>
      </td>
      <td valign=top>
        OK<br>
      </td>
    </tr>
    <tr>
      <td width=50%>
        John ate the fish peacefully in the park with a fork
      </td>
      <td width=50%>
        OK
      </td>
    </tr>
    <tr>
      <td valign=top>
        <font size=3>John ate the fish with a fork </font><font size=3>peacefully in the park</font><br>
      </td>
      <td valign=top>
        NOT OK<br>
      </td>
    </tr>
    <tr>
      <td width=50%>
        John ate the fish with a fork in the park peacefully
      </td>
      <td width=50%>
        NOT OK
      </td>
    </tr>
    </tbody>
  </table>
  <font size=3><br>
  </font><u><br>
  </u>
</div>
<div>
  <font size=4><b><u>Future Work</u></b></font>
</div>
<font size=3>&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;The SemanticSQL system works well within the bounds of the current grammar constructs it supports. </font><font size=3>However, developing a translation tool that is able to parse even a small subset of the entire English language is a daunting task.&nbsp; The sections below discuss a number of ideas that we believe would be useful expansions to SemanticSQL.<br>
<br>
</font>
<div>
  <u><font size=3><b>Multiple Prepositional Phrases and Adverbs</b></font></u>
</div>
<font size=3>&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;The current implementation only allows a single occurrence of location, adverb, and instrument phrases. One way to improve the system would be to move these properties of events to their own tables. &nbsp;This would allow a variable number of each to affect events once the proper grammar expansions were incorporated. &nbsp;This feature would allow sentences such as "John slept under the tree in the park" to be understood, and the question "Where did John sleep" would return "under the tree in the park".<br>
<br>
</font>
<div>
  <u><b><font size=3>Symmetric Relations</font></b></u>
</div>
<font size=3>&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;Currently, the system does not account for symmetric relations.&nbsp; For example, if "John kissed Mary", the current implementation does not know that "Mary kissed John".&nbsp; Adding symmetric relations would improve the system, especially when users ask questions based on those relations.</font>&nbsp; For example, the question "Did Mary kiss John" should return yes if "John kissed Mary" was submitted to the system.<br>
<b><br>
</b>
<div>
  <u><font size=3><b>Kimmo</b></font></u>
</div>
<div>
  <font size=3>&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;The system could benefit greatly from a closer connection to Kimmo and morphological rules. &nbsp;By incorporating morphological rules, the grammar could be easily expanded. &nbsp;A major advantage of this would be the ability to draw features from the vocabulary. &nbsp;For example, instead of storing "the dogs" as an object, all nouns could be broken down to their core meanings, such as "dog[plural, definite]". &nbsp;This would help with the indefinite object problem previously mentioned.&nbsp; Ideally, the Kimmo lexicon could be directly connected to the SQL vocabulary tables that have already been implemented.</font>
</div>
<br>
<div>
  <u><font size=3><b>If-Then Expansion</b></font></u>
</div>
<font size=3>&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;The current implementation of the "If...Then..." cause and effect rules is fairly primitive. &nbsp;The construct is parsed, acted upon, and then discarded. &nbsp;A more robust system could create a new "Rules" table that could store causal information. &nbsp;Then whenever an event occurs, the event could be checked against the rules table to see if any "effects" should be triggered.&nbsp; If one were to incorporate this expansion, he/she would have to consider whether or not&nbsp; he/she wanted later events to trigger "effects" that were previously entered. </font><br>
<br>
<div>
  <u><b><font size=3>Comparisons</font></b></u>
</div>
<font size=3>&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;An expansion to the project would be to add comparisons to the system.&nbsp; This would consist of adding quantities, such as "The red ball weighs 10 pounds", in addition to modifiers such as "little"and "big" that could be used in statements such as "The little dog walked in the park".&nbsp; As such, users would be able to ask questions such as "Which ball is the biggest" an<font size=3>d</font> "Does the re<font size=3>d</font> ball weigh more than the blue ball".<br>
<br>
<u><b>Time</b></u><br>
</font><font size=3>&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;Another expansion on this project would be to add the capability of time.&nbsp; For example, it would be a good feature for users to be able to ask questions such as "When did John go to sleep".&nbsp; Also, this could be expanded further by relating events based on time, such as "Did Tom go to the store before Mary came home".&nbsp; </font><br>
<br>
<div>
  <font size=3><u><b>Probabilistic Parsing</b></u></font><br>
  <div>
    &nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;A final powerful tool to incorporate into the project would be the addition of a probabilistic sentence parser such as a grammar trained on a large corpus like the Wall Street Journal. &nbsp;For example using the MXPOST maximum entropy tagger could allow the system to intelligently identify the part-of-speech of previously unknown words and appropriately add them to the dictionary database.&nbsp;<br>
    <font size=3><br>
    <br>
    </font>
  </div>
  <div>
    <font size=4><b><u>Contributions</u></b></font>
  </div>
  <div>
    <font size=3>In this SemanticSQL project we have:<br>
    </font>
  </div>
  <ul>
    <li>
      <font size=3>Expanded the English to SQL parser to incorporate new knowledge types: locations, instruments, negations, and passive phrases.</font>
    </li>
    <li>
      <font size=3>Added important grammar and semantic features for understanding including the idea of object classes.</font>
    </li>
    <li>
      <font size=3>Implemented new infrastructure to allow dealing with causal relations, indefinite constructs and an expanded lexicon.</font>
    </li>
    <li>
      <font size=3>Identified future areas for development on the English to SQL parser.</font>
    </li>
  </ul>
  <br>
  <br>
  <div>
    <font size=4><b><u>Appendix A: Example Sentences</u></b></font>
  </div>
  <div>
    <font size=3>The table below lists a number of sentences that can be used in the SemanticSQL system.</font>
  </div>
  <br>
  <table border=1 bordercolor=#000000 cellpadding=3 cellspacing=0 id=u_op>
    <tbody>
    <tr>
      <td>
        <font size=2>John slept under the tree</font>
      </td>
      <td>
        <font size=2>John ate the green moldy bread with the werewolf<br>
        </font>
      </td>
    </tr>
    <tr>
      <td>
        <font size=2>Poirot sadly gave the cats to Mary</font>
      </td>
      <td>
        <font size=2>What did John eat</font>
      </td>
    </tr>
    <tr>
      <td>
        <font size=2>Mary ate the icecream</font>
      </td>
      <td>
        <font size=2>What did John eat with</font>
      </td>
    </tr>
    <tr>
      <td>
        <font size=2>Where did John sleep</font>
      </td>
      <td>
        <font size=2>How did Fido toss the ball</font>
      </td>
    </tr>
    <tr>
      <td>
        <font size=2>Who gave the cats to Mary</font>
      </td>
      <td>
        <font size=2>Fido gracefully tossed the ball to Fido in the park with a glove<br>
        </font>
      </td>
    </tr>
    <tr>
      <td>
        <font size=2>Who did Poirot give the cats to</font>
      </td>
      <td>
        <font size=2><font size=2>Poirot slept</font><br>
        </font>
      </td>
    </tr>
    <tr>
      <td>
        <font size=2><font size=2>Mary likes John</font><br>
        </font>
      </td>
      <td>
        <font size=2>Where did Bill kill Mary</font><br>
      </td>
    </tr>
    <tr>
      <td>
        <font size=2>Jane ate icecream quickly in the park with a spoon<br>
        </font>
      </td>
      <td>
        <font size=2>If Poirot kissed Mary then Mary kissed Poirot</font>
      </td>
    </tr>
    </tbody>
  </table>
  <br>
  <br>
  <div>
    <font size=4><b><u>Appendix B: Technical Requirements and How to Run it</u></b></font><br>
  </div>
  <div>
    <font size=3>Getting SemanticSQL running can take a bit of work.&nbsp; The system has been tested using the following programs and libraries:</font>
  </div>
  <ul>
    <li>
      <font size=3>Python 2.6.5</font>
    </li>
    <li>
      <font size=3>NLTK 1.2</font>
    </li>
    <li>
      <font size=3>NLTK-Contrib 1.4.2</font>
    </li>
    <li>
      <font size=3>MySQLdb 1.2.3c1 (Python Library)</font>
    </li>
    <li>
      <font size=3>Apache 2.2.11</font>
    </li>
    <li>
      <font size=3>MySQL 5.1.36</font>
    </li>
  </ul>
</div>
<div>
  &nbsp;
</div>
<div>
  <font size=3>Once these are installed, Apache needs to be configured to run Python code as a CGI script.&nbsp;<br>
  </font><font size=3>The lines below should be present in the http.d file for apache.</font>
</div>
<div>
  <font size=3>&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;Options ExecCGI</font>
</div>
<div>
  <font size=3>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; AddHandler cgi-script .py</font>
</div>
<div>
  &nbsp;
</div>
<div>
  <font size=3>On linux, Apache can use the Shebang line of the python scripts to find the interpreter.</font>
</div>
<div>
  <font size=3>On windows, the following line can also be added to the http.d file to use the windows registry to find the interpreter.</font>
</div>
<div>
  <font size=3>&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;ScriptInterpreterSource Registry-Strict&nbsp;<br>
  <br>
  </font>
</div>
<div>
  <font size=3>In the main directory of the SemanticSQL project, the ".htaccess" file should contain the following lines, the PYTHON path should be modified appropriately:</font>
</div>
<div>
  <font size=3>&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;SetEnv PYTHONPATH C:/wamp/www/nlp/semantics/</font>
</div>
<div>
  <div>
    <font size=3>&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;Options ExecCGI</font>
  </div>
  <div>
    <font size=3>&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;AddHandler cgi-script .py<br>
    <br>
    </font>
  </div>
  <font size=3>MySQL will need to be setup to allow connections from the Apache server and should have the database from the nlp.sql (in the source code) available. &nbsp;The index.py file will need to be augmented with the SQL server information (user, pass, db, etc).<br>
  <br>
  </font><font size=3>Once this is done, SemanticSQL should be able to be run from the browser by accessing the index.py file.</font>
</div>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br></body>
</html>